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[57] ABSTRACT 

A method and system for storing a continuous feed of video 
is provided. According to one aspect of the invention, the 
continuous feed is encoded in a digital video format to 
produce a digital data stream. A series of content files is 
created by repeatedly performing the steps of (1) storing the 
digital data stream in a current file, and (2) establishing a 
new file as the current file when the current file satisfies a 
predetermined condition. If the series of content files con- 
tains more than a predetermined amount of the continuous 
feed, the oldest content file in the series of content files is 
deleted. Tag information that indicates information about 
frames contained in the digital data stream is generated. The 
tag information includes timestamps that indicate timing of 
frames relative to a beginning of the digital data stream. An 
initial time value that indicates an absolute time that corre- 
sponds to the beginning of the digital data stream. When a 
request from a client for playback beginning at a specified 
absolute time is received, the initial time value is subtracted 
from the specified absolute time to determine a relative time. 
The tag information is used to identify a location in the 
digital data stream that corresponds to the relative time. The 
digital data stream is then transmitted to the client beginning 
at the location in the digital data stream that corresponds to 
the relative time. 

20 Claims, 8 Drawing Sheets 
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METHOD AND APPARATUS FOR 
IMPLEMENTING SEAMLESS PLAYBACK 
OF CONTINUOUS MEDIA FEEDS 

RELATED APPLICATIONS 

This application is a continuation-in-part of U.S. patent 
application Ser. No. 08/859,860 which was filed on May 21, 
1997 and issued as U.S. Pat. No. 5,864,682 on Jan. 26, 1999, 
and which is a continuation of U.S. Patent Application Ser. 
No. 08,502,480 which was filed July 14, 1995 and issued as 
U.S. Pat. No. 5,659,539 on Aug. 19, 1997. 

The present application is related to: U.S. patent applica- 
tion Ser. No. 08/956,261, entitled "METHOD AND APPA- 
RATUS FOR CONCURRENTLY ENCODING AND TAG- 
GING MEDIA" filed by Daniel Weaver, Mark A. Porter and 
David J. Pawson", on the equal day herewith, the contents 
of which are incorporated herein by reference. 

U.S. patent application Ser. No. 08/956,263, entitled 
"METHOD AND APPARATUS FOR NON-SEQUENTIAL 
ACCESS TO AN IN-PROGRESS VIDEO FEED" filed by 
Daniel Weaver, Mark A. Porter and David J. Pawson on 
the equal day herewith, the contents of which are incorpo- 
rated herein by reference. 

FIELD OF THE INVENTION 

The present invention relates to a method and apparatus 
for processing audio-visual information, and more 
specifically, to a method and apparatus for providing non- 
sequential access to audio-visual information represented in 
a live content stream. 

BACKGROUND OF THE INVENTION 

In recent years, the media industry has expanded its 
horizons beyond traditional analog technologies. Audio, 
photographs, and even feature films are now being recorded 
or converted into digital formats. To encourage compatibil- 
ity between products, standard formats have been developed 
in many of the media categories. 

As would be expected, the viewers of digital video desire 
the same functionality from the providers of digital video as 
they now enjoy while watching analog video tapes on video 
cassette recorders. For example, viewers want to be able to 
make the video jump ahead, jump back, fast forward, fast 
rewind, slow forward, slow rewind and freeze frame. 

Various approaches have been developed to provide non- 
sequential playback of digital video data. With respect to 
digital video data, non-sequential playback refers to any 
playback operation that does not play all of the encoded 
frames in the exact order in the sequence in which they were 
encoded. For example, jump ahead and fast forward opera- 
tions are non-sequential in that some frames are skipped. 
Rewind operations at any speed are non-sequential in that 
during a rewind operation, frames are not played in the 
sequence in which they are encoded. 

One approach to providing non-sequential playback of 
digital video data, referred to herein as the tag-based 
approach, is described in U.S. Pat. No. 5,659,539, entitled 
"Method and Apparatus for Frame Accurate Access of 
Digital Audio-visual Information" issued to Porter et al on 
Aug. 19, 1997, the contents of which are incorporated herein 
by this reference. According to the lag-based approach, a 
stored digital video file is parsed to generate "tag informa- 
tion" about individual frames within the file. 

Specifically, the tag file contains information about the 
state of one or more state machines that are used to decode 
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the digital representation. The state information varies 
depending on the specific technique used to encode the 
audio-visual work. For MPEG-2 files, for example, the tag 
file includes information about the state of the program 

5 elementary stream state machine, the video state machine, 
and the transport layer state machine. 

During the performance of the audio-visual work, data 
from the digital representation is sent from a video pump to 
a decoder. The information in the tag file is used to perform 

30 seek, fast forward, fast rewind, slow forward and slow 
rewind operations during the performance of the audio- 
visual work. Seek operations are performed by causing the 
video pump to stop transmitting data from the current 
position in the digital representation, and to start transmit- 

35 ting data from a new position in the digital representation. 
The information in the tag file is inspected to determine the 
new position from which to start transmitting data. To ensure 
that the data stream transmitted by the video pump maintains 
compliance with the applicable video format, prefix data that 

20 includes appropriate header information is transmitted by 
the video pump prior to transmitting data from the new 
position. 

Fast forward, fast rewind, slow forward and slow rewind 
operations are performed by selecting video frames based on 

25 the information contained in the tag file and the desired 
presentation rate, and generating a data stream containing 
data that represents the selected video frames. The selection 
process takes into account a variety of factors, including the 
data transfer rate of the channel on which the data is to be 

30 sent, the frame type of the frames, a minimum padding rate, 
and the possibility of a buffer overflow on the decoder. 
Prefix and suffix data are inserted into the transmitted data 
stream before and after the data for each frame in order to 
maintain compliance with the data stream format expected 

35 by the decoder. 

The tag-based approach works well when there is enough 
time between the creation of the original digital video stream 
and the viewing of the digital video stream to allow the 
original digital video stream to be parsed to generate tag 

40 information. However, when the digital video stream is 
being viewed as it is being generated, parsing the digital 
video stream becomes impractical. The amount of compu- 
tational power required to parse the digital video stream as 
it arrives would be prohibitively expensive. On the other 

45 hand, it is not considered acceptable to increase the latency 
between the occurrence of many types of video feeds (e.g. 
sporting events) and the time at which such feeds are 
available for audience viewing. 
When a video stream is made available for viewing before 

50 generation of the stream has been completed, the video 
stream is said to be a "live feed". At a professional level, 
non-linear digital editors can be used to rapidly review 
footage of a live feed for a single user. However, these 
systems are not intended for and cannot be easily adapted to 

55 serve many users. For example, if a hundred users were 
watching the same live feed but wanted to rewind, pause, 
and fast forward the feed at different times, each would 
require a separate non-linear digital editor. 
Another problem associated with providing non-linear 

60 access to live digital video streams is that users may attempt 
to fast forward into portions of the video stream that do not 
yet exist. For example, a viewer may attempt to fast forward 
a live feed to see the final score of game which, in reality, 
has not yet ended. It is desirable to provide techniques for 

65 handling these types of situations in a way that ensures that 
the decoder will not freeze nor the video stream become 
corrupted. 
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Based on the foregoing, it is clearly desirable to provide FIG. 5 is a block diagram illustrating the migration of tag 

a method and apparatus for sequentially displaying non- information from an old tag file to a new tag file in response 

sequential frames of a live digital video. It is further desir- to the expiration of tag data within the old tag file, 
able to provide such non-sequential access to live digital 

video in a way that does not require each viewer to operate 5 DETAILED DESCRIPTION OF THE 

prohibitively expensive hardware. It is also desirable to PREFERRED EMBODIMENT 

provide safeguards against attempts to access portions of a A , . c ... ... 

f. % * . A method and apparatus for providing non-sequential 

live digital video stream that do not yet exist. , ...f, , . j , . 

^ 3 access to a live digital video stream is described. In the 

SUMMARY OF THE INVENTION following description, for the purposes of explanation, 

A .u j j . r . • c j c 10 numerous specific details are set forth in order to provide a 

A method and system for storing a continuous feed of u *\ . - . . \. .„ . 

... j , A • . & A , 4 . . . thorough understanding of the present invention. It will be 

video is provided. According to one aspect of the invention, . , t b , .„ *\ . iL . t tI _ 

4l _ J c ,. 5 j- j * i * j c , / apparent, however, to one skilled m the art that the present 

the continuous feed is encoded in a digital video format to . rr . . ... ... 4 . u . c . ; ., f 

, i • ■* i j * t * * r . * ni mvention may be practiced without these specific details. In 

produce a digital data stream. A senes of content files is tU . , n i . j j • 

* j i_ . r ... c \ * • iL other instances, well-known structures and devices are 

created by repeatedly performing the steps of (1) storing the is . . , \- c j . j -i 

, , , \ . ,° C1 j /-»\ * uv u* shown in block diagram form m order to avoid unnecessarily 

digital data stream in a current file, and (2) establishing a , . iL ^ . J 

a y .x. * £i t_ v ' C1 . £ obscuring the present invention, 

new file as the current file when the current file satisfies a or 

predetermined condition. If the series of content files con- FUNCTIONAL OVERVIEW 
tains more than a predetermined amount of the continuous 

feed, the oldest content file in the series of content files is 20 According to one aspect of the invention, the difficulty 

deleted. associated with applying the tag-based approach to live 

According to another aspect of the invention, the deletion digital video feeds is addressed by eliminating the need to 

of the oldest content file may be delayed if it is currently P arse an incoming digital video stream in real time. Instead 

being accessed. The deletion may be delayed until the oldest of generating tag data by parsing the digital video stream, 

content file is no longer accessed, or until a certain amount 25 the unit responsible for encoding the live feed retains 

of time has passed since the time at which it would have information about how the data was encoded and transmits 

been deleted but for the fact that it was being accessed. that information to the video server along with the encoded 

According to another aspect of the invention, tag infor- data ' ^ ta S information arrives at the video «*ver along 

mation that indicates information about frames contained in wth the corresponding content, so the content itself does not 

the digital data stream is generated. The tag information 30 have t0 be P arsed * 

includes timestamps that indicate timing of frames relative According to another aspect of the invention, the video 

to a beginning of the digital data stream. An initial time server is configured to ensure that the client cannot seek or 

value that indicates an absolute time that corresponds to the scan past the end of the received content. Due to the fact that 

beginning of the digital data stream. tnere w iH De some amount of skew between the arrival time 

When a request from a client for playback beginning at a 35 of tne content and the corresponding tags, the server is 

specified absolute time is received, the initial time value is configured to make sure that tags are not used prematurely, 

subtracted from the specified absolute time to determine a i e - such that the y would ca ^se the server to go past the end 

relative time. The tag information is used to identify a of tne mailable content, 

location in the digital data stream that corresponds to the pycxjidi adv ovctcm 

relative time. The digital data stream is then transmitted to 40 cAbMrLAKi ^Y^ibiw 

the client beginning at the location in the digital data stream FIG. 1 is a block diagram illustrating an exemplary 

that corresponds to the relative time. audio-visual information delivery system 100 for delivering 

BRIEF DESCRIPTION OF THE DRAWINGS and P rovidm S non-sequential access to live digital video 

feeds. Audio-visual information delivery system 100 gener- 

The present invention is illustrated by way of example, 45 ally mc i u des an encoder 101, a video server 106, a Media 

and not by way of limitation, in the figures of the accom- Da ta Store (MDS) 112, a database 116, a stream server 118. 

panying drawings and in which like reference numerals refer a video pump 12 o, and a client 122. 
to similar elements and in which: 

FIG. 1 is a block diagram that illustrated a video delivery THE ENCODER 

system according to an embodiment of the invention; 50 Encoder m md[Q ^ . and ates a 

MPETMH ' ^ atCS 1 dlgilal Stfeam ° f daU that enC ° deS the aUdi ° ViSUal in ? Ut 

an MFbu me, according to a particular format. Numerous video encoding 

FIG. 2B is a block diagram of an exemplary tag file formats have been developed and are well known in the 

according to an embodiment of the invention; 55 industry. F or example, the MPEG formats are described in 

FIG. 2C is a block diagram illustrating the tag information detail in the following international standards: ISO/IEC 

generated for each frame in an MPEG-1 file according to an 13818-1, 2, 3 (MPEG-2) and ISO/IEC 11172-1, 2, 3 

embodiment of the invention; (MPEG-1). Documents that describe these standards 

FIG. 3 A is a block diagram illustrating a storage system (hereafter referred to as the "MPEG specifications") are 

that uses RAID error correction techniques according to an 60 available from ISO/IEC Copyright Office Case Postale 56, 

embodiment of the invention; CH 1211, Geneve 20, Switzerland. While specific formats 

FIG. 3B is a block diagram illustrating a storage system may be referenced herein for the purposes of explanation, 

that combines RAID error correction and disk striping me present invention is not restricted to any particular digital 

according to an embodiment of the invention; stream format. 

FIG. 4 is a block diagram illustrating a series of content 65 Encoder 101 includes a Coder/Decoder (CODEC) 102 

files used to store the content of a continuous feed according and a multiplexer (MUX ) 104. CODEC 102 converts visual 

to an embodiment of the invention; and or audio-visual information from an input source to com- 
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pressed digital data. CODEC 102 may be, for example, a reliable protocol. For example, TCP is sufficient on lightly 

fractal compressor or an MPEG compressor. For the pur- loaded networks. The control information and metadata on 

poses of illustration, it shall be assumed that the video communication channel 130 contain more complicated data 

source being captured by CODEC 102 is a live source and, types and are sent via an object oriented protocol, such as the 

consequently, CODEC 102 is encoding video at IX relative 5 Common Object Resource Broker Architecture Interface 

to real time. However, the video source may alternatively be Definition Language ("CORBA IDL")- 

a stored video source which CODEC 102 encodes at any rate Communication between the encoder 101 and the video 

relative to real time. server 106 occurs in sessions. According to one 

MUX 104 multiplexes the compressed audio and visual embodiment, a session is performed in three phases: OPEN, 

information generated by CODEC 102 to generate a com- 10 SEND and CLOSE. The operations performed during each 

pressed video stream. In the compressed video stream, the of the phases is as follows: 

data representing video frames and audio are merged and OPEN — any provisioning that the video server 106 needs 

formatted according to the particular digital format sup- to perform for network or disk bandwidth or disk space 

ported by encoder 101. The specific operations performed occurs. A pipe for the video stream data (the "content") is 

during the merging process will vary based on the type of 15 created. 

encoding employed. For example, the merging process may SEND TAGS and SEND DATA— these calls are made 
involve determining the order and placement of portions of multiple times as content is encoded. The video server 106 
digitized audio and video in the stream and inserting meta- stores a u coate nt immediately to disk and updates an end- 
data at various points within the stream. The metadata may 0 f. me position. Tags are held in memory until the accom- 
take the form, for example, of header information that 20 panyin g conte nt data has been stored. Tags are held for an 
identifies the starting point and content of "packets" within additional period of time to guarantee that a seek to that tag 
the stream. The stream of compressed audiovisual informa- wi n succee d, i.e. that video pump 120 will not starve for 
tion constructed by MUX 104 is transmitted from the data 

encoder 101 to the video server 106 over a communication ^ CLOSE-content pipe is torn down. Server resources are 

channel 1ZS. released and content services and clients are notified that the 

CONTROL INFORMATION ^ eec ^ nas Decome a normal static piece of content. 

Encoder 101 generates content data and control data in 
According to one aspect of the invention, the encoder 101 parallel. However, the control data associated with a par- 
sends control information to the video server 106 over a 3o ticular porlion of content i s nol necessarily generated by 
communication channel 130 in parallel with the video encoder 101 at the same time as the particular portion of 
stream. The control information sent over channel 130 content. For example, encoder 101 may actually determine 
includes specific information about how the encoder 101 how it ^ going t0 line up content frames before the encoder 
constructed the video stream. This control information 101 actually lines up the frames. Under these circumstances, 
includes tag data that will be used by the stream server 118 ^ the control data that indicates how the frames are lined up 
to provide non-sequential access to the video stream. may be transmitted by encoder 101 before the content data 
Specifically, the control information may include informa- t hat contains the frames, 
tion about the type, length, and boundaries of the various 

frames encoded in the video stream as well as header THE VIDEO SERVER 

information that specifies the compression ratio the bit rate, ^ VideQ SMVCr 106 ^ video stfeam and CQntrol 

and other types of information the video server 106 requires daU frQm CDCoder 1Q1 and 

causes the data to be stored in 

to determine how to process the video stream. MDS 112. In the illustrated system, the video server 106 

Significantly, the generation of the control information an MPEG video stream to MDS server 110, and MDS 

involves minimal additional computational power because server 110 stores the MPEG video stream in an MPEG file 

MUX 104 generates most of the information already during 45 134. m parallel, the video server 106 sends to the MDS 

the construction of the content stream. Specifically, MUX server 110 tag information extracted from the control data 

104 arranges and encapsulates the digital video and audio received on line 130. The tag data is stored in a tag file 132 

data from CODEC 102. Since MUX 104 is packaging the on disks 114. The video server 106 may also send informa- 

content, MUX 104 knows the contents of and boundaries tion about the content, including tag data, to be stored in 

between the packages. 50 database 116. 

COMMUNICATION BETWEEN THE ENCODER 0nce ta & data * fr an smi tted b y video se ™ er 1° 6 > aQ y 

AND THE VIDEO SERVER ot ^ er entltv 1D l " e svstem » including video pump 120, may 

use the tag data to attempt to access the content associated 

While CODEC 102 will typically be implemented in with the tag data. Consequently, the immediate transmission 

hard-wired circuitry, MUX 104 is preferably implemented 55 0 f tag data to MDS server 110 may result in errors when, for 

by program-controlled circuitry, such as a processor pro- example, the tag data arrives at video server 106 before the 

grammed to execute a particular sequence of instructions corresponding content data. Therefore, prior to sending the 

that are stored in a memory. Consequently, MUX 104 may tag data to MDS server 110, video server 106 buffers each 

include a processor executing a conventional multiplexing tag data item in a tag buffer 108 until it is safe for entities, 

program that has been linked with and makes calls to a 6 q suc h as video pump 120, to access the content associated 

software library that controls the communication with the with the tag data item. The use of tag buffer 108 to avoid 

video server 106. premature reads of content data is described in greater detail 

All data transmitted between the encoder 101 and the hereafter, 

video server 106 is preferably sent using a reliable commu- pypmpt ady mppp ptf p 

nication mechanism. According to one embodiment, the 65 cAcMr lak i Mrcu rii_n 

video content on communication channel 128 is handled as Digital audio- visual storage formats, whether compressed 

a simple bytestream and is transmitted via a lightweight, or not, use state machines and packets of various structures. 



05/12/2003, EAST Version: 1.03.0002 



EXEMPLARY TAG FILE 



6,138,147 

7 8 

The techniques described herein apply to all such storage and techniques are incorporated in the tag generator and the 

formats. While the present invention is not limited to any stream server. All of the other elements of the server are 

particular digital audio-visual format, the MPEG-2 transport format independent, 
file structure shall be described for the purposes of illustra- 
tion. 

Referring to FIG. 2a, it illustrates the structure of an 

MPEG-2 transport file 134 in greater detail. The data within The contents of an exemplary tag file 132 shall now be 

MPEG file 134 is packaged into three layers: a program described wiLh reference to FIG. 2b. In FIG. 2b, the tag file 

elementary stream ("PES") layer, a transport layer, and a 132 includes a file type identifier 202, a length indicator 204, 

*™ e ^ yer * ^ eSe . layC ? tl^ 6 T.^J 0 ^ a bit rate indicator 206, a play duration indicator 208, a 

MPEG-2 specification* M the PES layer MPEG file 134 frame Qumber 210 ^ m access information 212 

l 0tlS M ppr a n^f < traD r P ° rt yef ; and aa initial MPEG time oflset 213. File type identifier 202 

the MPEG file 134 consists of a sequence of transport ..... , • wnr^. +<*a ^ 

packets. At the video layer, MPEG file 134 consists of a indicat f s *? P^ slca ' ™>W m * n ° n lhe MPE , G file 13 * *> r 

sequence of picture packets. Each picture packet contains is ™^ k ' ' J^ w^". 7,n^ *f " 

the data for one frame of video. MPEG file 134 is a MPEG-2 or an MPEG-1 file. 

Each PES packet has a header that identifies the length Length indicator 204 indicates the length of the MPEG 

and contents of the PES packet. In the illustrated example, file 134. Bit rate indicator 206 indicates the bit rate at which 

a PES packet 250 contains a header 248 followed by a the contents of the MPEG file 134 should be sent to a client 

sequence of transport packets 251-262. PES packet bound- 20 during playback. The play duration indicator 208 specifies, 

aries coincide with valid transport packet boundaries. Each i n milliseconds, the amount of time required to play back the 

transport packet contains exclusively one type of data. In the entire contents of MPEG file 134 during a normal playback 

illustrated example, transport packets 251, 256, 258, 259, operation. Frame number indicator 210 indicates the total 

260 and 262 contain video data. Transport packets 252, 257 number of frames presented in MPEG file 134. 
and 261 contain audio data. Transport packet 253 contains 

control data. Transport packet 254 contains timing data. Stream access information 212 is information required to 

Transport packet 255 is a padding packet. access the video and audio streams stored within MPEG file 

Each transport packet has a header. The header includes a 134. Stream access information 212 includes a video 

program ID ("PID") for the packet. Packets assigned PID 0 3Q elementary stream ID and an audio elementary stream ID. 

are control packets. For example, packet 253 may be For MPEG-2 files, stream access information 212 also 

assigned PID 0. Other packets, including other control includes a video PID and an audio PID. The tag file header 

packets, are referenced in the PID 0 packets. Specifically, may also contain other information that may be used to 

PID 0 control packets include tables that indicate the packet implement other features, 

types of the packets that immediately follow the PID 0 „ _ . _ , . r . J ., , . 

control packets. For all packets which are not PID 0 control 35 In ad * ll °° J i0 lhe ? eneral ^formation described above, 

packets, the headers contain PIDs which serve as a pointers the ta S file 132 00013105 ao entr y for cach frame withia lhe 

into the table contained in the PID 0 control packet that most MPEG file 134 - ^ entr y for a vldeo frame includes 

immediately preceded the packets. For example, the type of information about the state of the various MPEG layers 

data contained in a packet with a PID 100 would be relative to the position of the data that represents the frame, 

determined by inspecting the entry associated with PID 100 For an MPEG-2 file, each entry includes the state of the 

in the table of the PID 0 control packet that most recently MPEG-2 transport state machine, the state of the program 

preceded the packet. elementary stream state machine and the state of the video 

In the video layer, the MPEG file 134 is divided according state machine. For an MPEG-1 file, each entry includes the 

to the boundaries of frame data. As mentioned above, there 45 current state of the Pack system MPEG stream and the state 

in no correlation between the boundaries of the data that of the video state machine. 

represent video frames and the transport packet boundaries. ^ fi1 , ... t , . . , . . , 

j ,i . * j , t_ r _, c ' j Tag file entry 214 illustrates in greater detail the tag 

In the illustrated example, the frame data for one video • e 4 , : ' . , , c . . , . %Jtnr ^ -> -j 

frame «F» is located as indicated by brackets 270. f° T ™T ^ * St ° red f °/ " ^idual MPEG-2 video 

Specifically, the frame data for frame «F" is located from a 50 frame F ' Wlth reSpeC u l t0 th ? State ° f the P^ am C \ Gm T 

point 280 within video packet 251 to the end of video packet ta ? slream sta j e maclune ' lhe ta 8 cntr y 214 mcludes the 

251, in video packet 256, and from the beginning of video "^formation mdicated in Table 1. 
packet 258 to a point 282 within video packet 258. 

Therefore, points 280 and 282 represent the boundaries for TABLE 1 

the picture packet for frame "F" The frame data for a second 55 DAIA meaning 
video frame "G" is located as indicated by brackets 272. The 



boundaries for the picture packet for frame "G" are indicated PES OFFSET at the start of The offset, within the pes packet 

bv bracket 276 PICTURE 217 that contains the frame data for 

y frame **F' of the first byte 

Structures analogous to those described above for of the frame data for frame M F\ 
MPEG-2 transport streams also exist in other digital audio- 60 PES offset at the end of The offset between the last byte in 

visual storage formats, including MPEG-1, Quicklime, AVI, PICnjRE 2 ^ the frame data for frame -f- 

n- u j 11 c . w . * r j i_ / and the end of the PES packet in 

Proshare and H.261 formats. In the preferred embodiment, which ^ tnXM dala fo ; frame 

indicators of video access points, time stamps, file locations, **F' resides. 

etc. are stored such that multiple digital audio-visual storage " — — 

formats can be accessed by the same server to simulta- 65 

neously serve different clients from a wide variety of storage With respect to the state of the video state machine, tag 

formats. Preferably, all of the format specific information entry 214 includes the information indicated in Table 2. 
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TABLE 2 



TABLE 4 



DATA 



MEANING 



PICTURE SIZE 220 



START POSITION 226 



TIME VALUE 228 



FRAME TYPE 232 



TIMING BUFFER 
INFORMATION 238 



The size of the picture packet for frame 
M F\ 

The location within the MPEG file of 
the first byte of the data that corresponds 
to frame "F* 

The time, relative to the beginning of 
the movie, when frame "F* would be 
displayed during a normal playback of 
MPEG file 134. 

The technique used to encode the frame 
(e.g. I-frarne, P-frame or B-frame). 
Indicates how full the buffer of the 
decoder is (sent to the decoder to 
determine when information should be 
moved out of the buffer in order to 
receive newly arriving information). 



TABLE 3 



DATA 



MEANING 



START OFFSET 234 



# OF NON- VIDEO 
PACKETS 222 



# OF PADDING 
PACKETS 224 



END OFFSET 236 



CURRENT CONTINUITY 
COUNTER 215 
DISCONTINUITY 
FLAG 230 



The distance between the of the first 
byte in the frame data and the start of 
the transport packet in which the first 
byte resides. 

The number of non-video packets (i.e. 
audio packets, padding packets, control 
packets and timing packets) that are 
located within the picture packet for 
frame "F\ 

The number of padding packets that are 
located within the picture packet for 
frame "F\ 

The distance between the last byte in the 
frame data and the end of the packet in 
which the last byte resides. 
The Continuity value associated with 
frame "F\ 

Indicates whether there is a 
discontinuity in time between frame "F* 
and the frame represented in the 
previous tag entry. 



Assume, for example, that entry 214 is for the frame "F" 
of FIG. 2/>. The size 220 associated with frame "F" would 
be the bits encompassed by bracket 274. The number 222 of 
non-video packets would be five packets 252, 253, 254, 255 
and 257). The number 224 of padding packets would be one 
(packet 255). The start position 226 would be the distance 
between the beginning of MPEG file 134 and point 280. The 
start offset 234 would be the distance between the start of 
packet 251 and point 280. The end offset 236 would be the 
distance between point 282 and the end of packet 258. 



The tag information generated for each frame in an 
MPEG-1 file is illustrated in FIG. 2c. Referring to FIG. 2c, 
entry 214 includes data indicating the state of three state 
machines: a system state machine, a pack state machine, and 
a video state machine. Specifically, tag entry 214 includes 
the information shown in Table 4. 



DATA 



20 



With respect to the state of the transport layer state 
machine, tag entry 214 includes the information indicated in 
Table 3. 



MEANING 



5 AMOUNT OF NON- VIDEO 
DATA 221 



AMOUNT OF PADDING 
DATA 223 



10 



PACK OFFSET AT 
START 225 



15 



PACK REMAININO AT 
START 227 



PACK OFFSET AT 
END 229 



PACK REMAINING AT 
END 231 



25 PICTURE SIZE 233 

PICTURE START POS 235 

30 PICTURE END POS 237 

FRAME TYPE 239 
TIME VALUE 241 

35 

TIMING BUFFER INFO 243 

40 



The amount of non -video data (in bytes) 
contained within the start and end 
boundaries of the frame data for frame 
"F\ 

The amount of padding data (in bytes) 
contained within the start and end 
boundaries of the frame data for frame 
"F\ 

The offset between the start boundary of 
the frame data for frame "F* in the 
beginning of the pack packet that 
contains the start boundary for frame 
"F\ 

The distance between the start boundary 
for frame "F" and the end of the pack 
packet that contains the start boundary 
of frame "F\ 

The offset between the end boundary for 
frame "F' in the beginning of the packet 
that contains the end boundary for frame 
W F\ 

The distance between the end boundary 
for frame "F' and the end of the pack 
packet that contains the end boundary of 
frame "F\ 

The distance (in bytes) between the start 
boundary for frame "F" and the end 
boundary for frame "F\ 
The distance between the start of the 
MPEG-1 file and the start boundary for 
frame H F\ 

The position, relative to the beginning 
of the MPEG-1 file, of the end boundary 
for frame "F\ 

The technique used to encode the data 
that represents frame "F*. 
The time, relative to the beginning of 
the movie, when frame "F* would be 
displayed during a normal playback of 
MPEG file 134. 

Indicates how full the decoder is (sent to 
the decoder to determine when 
information should be moved out of the 
buffer in order to receive newly arriving 
information). 



The tag information includes data indicating the state of 
the relevant state machines at the beginning of video frames. 

45 However, the state machines employed by other digital 
audio-visual formats differ from those described above just 
as the state machines employed in the MPEG-1 format differ 
from those employed in MPEG-2. Consequently, the specific 
tag information stored for each frame of video will vary 

50 based on the digital audio-video format of the file to which 
it corresponds. 

THE MDS 



MDS 112 includes MDS server 1110 and one or more 
55 non-volatile storage devices, such as disks 114. In the 
illustrated embodiment, MPEG file 134 is stored across 
numerous disks 114 to increase the fault tolerance of the 
system. Consider, for example, the multi-disk system 300 
illustrated in FIG. 3. System 300 includes N+l chsk drives. 
60 An MPEG file is stored on N of the N+l disks. The MPEG 
file is divided into sections 350, 352, 354 and 356. Each 
section is divided into N blocks, where N is the number of 
disks that will be used to store the MPEG file. Each disk 
stores one block from a given section. 
65 In the illustrated example, the first section 350 of the 
MPEG file includes blocks 310, 312 and 314 stored on disks 
302, 304 and 306, respectively. The second section 352 
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includes blocks 316, 318 and 320 stored on disks 302, 304 bypassing RAID during fast forward and fast rewind 

and 306, respectively. The third section 354 includes blocks operations, disk bandwidth remains at the same level or 

322, 324 and 326 stored on disks 302, 304 and 306, below that used during normal playback operations, 
respectively. The fourth section 356 includes blocks 328, since RAID is not used during real-time fast forward and 

330 and 332 stored on disks 302, 304 and 306, respectively. 5 f ast rewind operations, faulty data cannot be reconstructed 

The disk 308 which is not used to store the MPEG file is during these operations. Consequently, when the video 

used to store check bits. Each set of check bits corresponds pump 120 detects that the data for a selected frame is 

to a section of the MPEG file and is constructed based on the corrupted or unavailable, the video pump 120 discards the 

various blocks that belong to the corresponding section. For entire segment associated with the problem frame. Thus, if 

example, check bits 334 corresponds to section 350 and is 10 the data associated with a frame cannot be sent, then the 

generated by performing an exclusive OR operation on all of prefix and suffix data for the frame is not sent either, 

the blocks in the first section 350. Similarly, check bits 336, However, any padding packets that were to be sent along 

338 and 340 are the products of an exclusive OR performed with the prefix or suffix data will still be sent, 
on all of the blocks in the section 352, 354 and 356, By sending data in entire "segments", conformance with 

respectively. 15 the digital audiovisual format is maintained. In one 

System 300 has a higher fault tolerance than a single disk embodiment, the video pump 120 will send down padding 

system in that if any disk in the system ceases to operate packets to fill the line to maintain the correct presentation 

correctly, the contents of the bad disk can be reconstructed rate. In the preferred embodiment, this behavior is selectable 

based on the contents of the remaining disks. For example, by the client, 
if disk 304 ceases to function, the contents of block 312 can 20 

be reconstructed based on the remaining blocks in section DATA STRIPING 

350 and the check bits 334 associated with section 350. ^ D techoi described above ^ ^ 

Similarly, block 318 can be constructed based on the remain- . . . n j • * n a- i • 

, , ; ■ i i * i t*i 11^" • j throughput (because all data from all disks in an array are 

me blocks in section 352 and the check bits 336 associated j- n A i- u r. /a * • \ 

>*x- **t a,- _i * i 4 , 25 read in parallel) and reliability (due to error correction). To 

with section 352. This error detection and correction tech- - 4l _ . ' , , t n ' A v m , , . . 

„ . „„ , . AA ri further increase throughput, RAID may be used in coni unc- 

mque is generally known as Redundant Array of I nexpen- . , . . • • n ■ a * \ ■ ■ \ c 

sive Disks" or RAID 100 h striping. Using data striping, segments of 

logically sequential data are written to multiple physical 

During real-time playback using RAID, video pump 120 devices (of sets of physical deyices) in a round . robin fash . 

reads and processes the MPEG file on a section by section 3Q ion ^ amount of data slored at each stQrage ekment in lhe 

basis so that all of the information is available to reconstruct round . robin sequence is referred to as a "stripe". When each 

any faulty data read from disk. Techniques for performing storage elemeQt in the roU nd-robin sequence is an array of 

RAID in real time are described in U.S. Pat. No. 5,623,595, RAID disks, each segment of data is referred to as a RAID 
entitled "METHOD AND APPARATUS FOR ^ B 

TRANSPARENT, REAL TIME RECONSTRUCTION OF „ _L* .„ t f , . ........ . . 

CORRUPTED DATA IN A REDUNDANT ARRAY DATA 35 . .' 3B P lustr ^ a A !^ l S? m whl ° h ^ £ Dg * T 

cTnnxrr cvcttu.. .u * * c u- u * • in conjunction with RAID. The sy stem of FIG. 3B is similar 
STORAGE SYSTEM , the contents of which is mcorpo- , J -„ T ~ „ . . , , J . . . - . . 

+ J1 . . ... P to that of FIG. 3 A with the exception that each of the disks 

rated herein by this reference, . ialu i au p\m a- i 

_ . , , , , • , «- . m rlG. 3A has been replaced by a series of M disks. Thus, 

During normal playback operations, there is sufficient 6iskm has been replaced bv disks 302-1 through 302-M. 

time to perform the disk accesses required to read an entire 40 ^ ^ dons stored on disk 302 have ^ s , ored on 

section while the data from the pre^ous section is being ^ 3^ , 0 m _ M {n a mial> round robin fashion . 

truamtf ted in the MPEG data stream. However during fast Fof u assume that ^ MpEG me has ^ divided 

forward and fast rewind operations less than aU of the data ^ 5fJ ^ and t|jat djsk 302 has beeQ laced with 

in any section will be sent m the MPEG data stream 2 5 disks. Under those circumstances, disk 302-1 would store 

Because less data is sent, the transmission of the data will 45 (he ^ jon of s ts x ^ 26 . Disk 302 .2 would 

take less time. Consequently, less time will be available to store tf]e flrsl jon of g , s 2 and 27 e , c 

read and process the subsequent section. ^ ... , 

, , . . . Data stnpmg mcreases throughput because different pro- 

For example assume that only one frame X from section cesses ^ ^ readin from differen , djsk ia m 

350 was selected for display durmg a fast forward operauon. Fof ^ Qne ^ be feadm ^ &m 

During the time it takes to transmit lhe segment for frame X, 50 M ent of an MPEG file &om the r^d array a,,, inc i udes 

the data tor the next selected tram e Y is read and processed. . + ~ l i i kt 1 u-i *u a * 

. . - . ,. . ^„ , Disk 1,1 through Disk 1,N+1, while another data pump is 

Assume that the next frame Y is located in section 352. If the 4 , °Z , . c # . wnr^ 

i*^^ £i • . « j . • . concurrently readme the second segment of the same MPEG 

MPEG me ^ read and processed on a secuon by secUon flle from ^ ^ ^ m | ludes ^ 

basis (required for RAID), then all of the blocks in section 2 

352 are read and processed during the transmission of the 55 ~ ' 

single frame X. Even if it were possible to read and process For .throughput performance reasons, reads and writes 

all of the blocks in section 352 in the allotted time, it may occur m d ! sc , rete chunks - tv P lcal| y disk ^ Io c ? 

still be undesirable to do so because of the resources that 'VP'" 1 d 'S" al Vlde0 svstem > each acc f 55 umt 18 256 

would be consumed in performing the requisite disk or 2 megabits, and the content is 2 Mb/sec MPEG, 

accesses Consequently, each RAID stripe corresponds to appro h- 

1 i- L* c * L c • * 1 i irv a . mately one second of video, though this could easily range 

In hght of the foregoing, video pump 120 does not use c J , 1rt , f- j j- . . 

«Atr» j • c r j a c a from about 0.2 to 10 seconds per stripe depending on content 

RAID dunng fast forward and fast rewind operations. ... . , - f. r r ° 

n • j j A* * i bit rate and server configuration. 

Rather, video pump 120 reads, processes and transmits only & 

the data indicated in the commands it receives from the THE CLIENT 

stream server 118. Thus, in the example given above, only 65 

the frame data for frame Y would be read and processed Audio-visual information delivery system 100 contains 

during the transmission of the segment for frame X. By one or more clients, such as client 122. Client 122 generally 
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represents devices configured to decode audio-visual infor- 
mation contained in a stream of digital audio-visual data. For 
example, client 122 may be a set top converter boxes 
coupled to an output display, such as television. Client 122 
includes a decoder 126 for decoding a digital data stream, 5 
and a control unit 124 for communicating information to the 
stream server 118. 

Stream server 118 is able to receive information from 
client 122 over a control network 140. Control network 140 
may be any network that allows communication between 30 
two or more devices. For example, control network 140 may 
be a high bandwidth network, an X.25 circuit or an elec- 
tronic industry association (EI A) 232 (RS-232) serial line. 

The client 122 communicates with the stream server 118 
and database 116 via the control network 140. For example, 
client 122 may send a query to database 116 requesting 
information about what is currently available for viewing. 
The database 116 responds by sending the requested infor- 
mation back to the client 122. The user of client 122 may 
then select to view a particular audio-visual work beginning 
at a particular location and at a particular speed. Client 122 
transmits requests to initiate the transmission of audio-visual 
data streams and control information to affect the playback 
of ongoing digital audio-visual transmissions through net- 
work 140 to stream server 118. 

THE VIDEO PUMP AND STREAM SERVER 

The video pump 120 is coupled to the stream server 118 
and receives commands from the stream server 118. 'file 3Q 
video pump 120 is coupled to the disks 114 such that the 
video pump 120 stores and retrieves data from the disks 114. 

In addition to communicating with the stream server 118, 
the client 122 receives information from the video pump 120 
through a high bandwidth network 150, The high bandwidth 35 
network 150 may be any of type of circuit-style network link 
capable of transferring large amounts of data. A circuit-style 
network link is configured such that the destination of the 
data is guaranteed by the underlying network, not by the 
transmission protocol. For example, the high bandwidth 40 
network 150 may be an asynchronous transfer mode (ATM) 
circuit or a physical type of line, such as a Tl or El line. In 
addition, the high bandwidth network 150 may utilize a fiber 
optic cable, twisted pair conductors, coaxial cable, or a 
wireless communication system, such as a microwave com- 45 
munication system. 

Network 150 may alternatively be a relatively low band- 
width network, or a combination of high bandwidth and low 
bandwidth communication mediums. For example, a portion 
of network 150 may comprise a relatively high bandwidth 50 
ATM circuit, while a relatively low bandwidth device such 
as a 28 .8K modem is used downstream to deliver the video 
information from the network to the client 122. 

The audio-visual information delivery system 100 permits 
a server, such as the video pump 120, to transfer large 55 
amounts of data from the disks 114 over the high bandwidth 
network 150 to the client 122 with minimal overhead. In 
addition, the audio- visual information delivery system 100 
permits the client 122 to transmit requests to the stream 
server 118 using a standard network protocol via the control 60 
network 140. In a preferred embodiment, the underlying 
protocol for the high bandwidth network 150 and the control 
network 140 is the same. The stream server 118 may consist 
of a single computer system, or may consist of a plurality of 
computing devices configured as servers. Similarly, the 65 
video pump 120 may consist of a single server device, or 
may include a plurality of such servers. 



To receive a digital audio-visual data stream from a 
particular digital audio-visual file, client 122 transmits a 
request to the stream server 118. In response to the request, 
the stream server 118 transmits commands to the video 
pump 120 to cause video pump 120 to transmit the requested 
digital audio-visual data stream to the client that requested 
the digital audio-visual data stream. For live feeds, the video 
server 106 will be storing the video stream into the video file 
134 at the same time the video pump 120 is sending a video 
stream from the file 134 to the client 122. 

The commands sent to the video pump 120 from the 
stream server 118 include control information specific to the 
client request. For example, the control information identi- 
fies the desired digital audio-visual file, the beginning offset 
of the desired data within the digital audio-visual file, and 
the address of the client. In order to create a valid digital 
audio- visual stream at the specified offset, the stream server 
118 also sends "prefix data" to the video pump 120 and 
requests the video pump 120 to send the prefix data to the 
client. As shall be described in greater detail hereafter, prefix 
data is data that prepares the client to receive digital audio- 
visual data from the specified location in the digital audio- 
visual file. 

The video pump 120, after receiving the commands and 
control information from the stream server 118, begins to 
retrieve digital audio-visual data from the specified location 
in the specified digital audio -visual file on the disks 114. For 
the purpose of explanation, it shall be assumed that audio- 
visual information delivery system 100 delivers audio-visual 
information in accordance with one or more of the MPEG 
formats. Consequently, video pump 120 will retrieve the 
audio -visual data from an MPEG file 134 on the disks 114. 

The video pump 120 transmits the prefix data to the client, 
and then seamlessly transmits MPEG data retrieved from the 
disks 114 beginning at the specified location to the client. 
The prefix data includes a packet header which, when 
followed by the MPEG data located at the specified position, 
creates an MPEG compliant transition packet. The data that 
follows the first packet is retrieved sequentially from the 
MPEG file 134, and will therefore constitute a series of 
MPEG compliant packets. The video pump 120 transmits 
these packets to the requesting client via the high bandwidth 
network 150. 

The requesting client receives the MPEG data stream, 
beginning with the prefix data. The client decodes the 
MPEG data stream to reproduce the audio-visual sequence 
represented in the MPEG data stream. 

PREMATURE READ AVOIDANCE 

When client 122 is playing an MPEG stream at the same 
time the MPEG stream is being generated by encoder 101, 
safeguards should be taken to ensure that client 122 docs not 
stall (because it has reached the end of valid content data) or 
play bad data (because it has read beyond the end of the 
currently available content data). If the video pump 120 
prematurely reads a stripe of disks 114, video pump 120 will 
send invalid data to the client 122, resulting in the display of 
unintended content or garbage (dirty content). Such a pre- 
mature read will occur if, for example, a user requests 
display of a portion of the video stream that has not yet been 
stored on disks 114. To prevent this, end-of-file information 
for MPEG file 134 is maintained to indicate the current 
end-of-file 134. As more content data is added to file 134, the 
end-of-file information is updated so that the new data may 
be accessed. 

One approach to avoid premature reads is to repeatedly 
update a table of contents on disks 114 with a new end-of- 
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file value, and have the video pump 120 check this value 
before reading stripes from disks 114. The MDS server 110 
updates the end-of-file to indicate that the content file 134 
includes new content only after it has been verified that the 
new content has been successfully stored to disks 114. 5 
Unfortunately, unless this end-of-file information is guaran- 
teed to be held in dynamic memory, this technique leads to 
a jitter in the latency period of updates that is difficult to 
predict. 

Another approach to avoid premature reads is for the 10 
MDS server 110 to actively transmit the new end-of-file 
information to all processes that are reading the content. 
Thus, MDS server 110 stores content data into file 134 on 
disks 114, waits for verification that the content has been 
stored, and then transmits messages indicating the existence 15 
of the newly stored content to all processes reading the 
content data (e.g. video pump 120). The MDS server 110 
may make such end-of-filc notification messages periodi- 
cally (e.g. after every 5 seconds) or after a predetermined 
amount of new content data has been successfully stored 20 
(e.g. after every 1 Megabyte). Unfortunately, notification 
times will also jitter due to variations in the content arrival 
times, which is a function of the encoder 101 and the 
network between the encoder 101 and the video server 106. 

According to one embodiment, the tag information is used 25 
to indicate the current end-of-file. Specifically, video server 
106 effectively updates the end-of-file of file 134 by sending 
tag information from tag buffer 108 for storage by MDS 112. 
As soon as the tag information that corresponds to a par- 
ticular portion of content has been transmitted by video 30 
server 106, the video pump 120 is free to perform a seek to 
that particular portion of video. Until the tag information 
that corresponds to a particular portion of video is released, 
video pump 120 may not perform a seek to the correspond- 
ing portion of video. Because the newest tag information 35 
indicates the current end-of-file, newly connected users may 
simply seek to the content associated with the newest tag 
information, and begin playing the feed at the real-time rate. 

MINIMUM TAG DELAY PERIOD 4 ° 

To prevent client 122 from stalling or playing bad data, 
the transmission of tag data from tag buffer 108 to MDS 112 
is delayed. Preferably, the duration of the delay is long 
enough to ensure that the associated content data will not be 45 
prematurely accessed. On the other hand, delaying the tag 
data longer than necessary increases the latency between 
when content is encoded and when users can seek or scan to 
the content. Consequently, it is desirable to determine a 
minimum tag delay period, and to buffer tag data in tag 50 
buffer 108 for the minimum tag delay period. The minimum 
tag delay period for a tag data item is determined by the 
maximum latency associated with delivering the corre- 
sponding content data from encoder 101 to video pump 120. 

Video server 106 includes a network buffer 152 and a 55 
write buffer 154. Typically, the video server 106 will be 
reading content data from channel 128 into network buffer 
152 at the same time that video server 106 is writing content 
data from write buffer 154 to disks 114. In embodiments that 
use RAID storage techniques, content data is received and go 
buffered within video server 106 in units that correspond to 
one RAID stripe. 

Video pump 120 includes a prefetch unit 146 and a buffer 
144. Video pump 120 reads content data from disks 114 
asynchronously. To read content data, prefetch unit 146 65 
requests the transmission of a particular portion of content 
data, and disks 114 respond by either sending the requested 
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content data or by indicating that the requested data cannot 
be sent. Some latency occurs between the time the prefetch 
unit 146 requests data, and the time the data is received by 
video pump 120. 

When content data from file 134 arrives at video pump 
120, video pump 120 stores the content data from file 134 
into the buffer 144. As bandwidth becomes available on 
network 150, video pump 120 transmits content data from 
the buffer 144 over network 150 to client 122. As with the 
video server 106, content data is pre-fetched and buffered 
within video pump 120 in units that correspond to one RAID 
stripe when RAID storage techniques are used. 

As explained above, the video pump 120 is typically 
copying data from one RAID stripe into network buffers and 
prefetching the following stripe. Likewise, the video server 
106 is typically writing one RAID stripe of content to the 
data store and receiving data from the network into a second 
memory buffer. Consequently, there are typically four RAID 
stripes "in transit", so the latency between when any content 
data is generated and when it is available to be played is 
approximately the time it takes to deliver four RAID stripes 
worth of data. 

RAID stripes are usually 128K bits or 256K bits per disk. 
The combined total of all disks in a RAID stripe is therefore 
1 to 2 Megabits. For typical MPEG files, each raid stripe will 
correspond to approximately one second of video. 
Consequently, having four RAID stripes in transit results in 
a minimum latency of approximately 4 seconds. 

The implication for tag data is that a given tag may only 
be released by the video server 106 for use by other entities 
when the corresponding content is available to be played 
(i.e. has been successfully stored on disk for two seconds). 
Therefore, in a video delivery system where the content 
delivery has a four second latency, the tag data retained in 
tag buffer 108 is transmitted no earlier than four seconds 
after the generation of the corresponding content. 

According to one embodiment, jitter and stalling are both 
avoided by transmitting a batch of tag data from tag buffer 
108 to MDS 112 every twelve seconds. The tag data batch 
transmitted at every twelve second interval includes all tag 
information in tag buffer 108 that is at least twelve seconds 
old. Tag data that is less than twelve seconds old is retained 
in tag buffer 108 and transmitted to MDS 112 in a batch at 
the end of the next twelve second interval. MDS server 110 
sends the tag data to the various entities (e.g. video pump 
120) that are reading video file 134, and then stores the tag 
information on disks 114. 

DIGITAL CHANNELS 

Video files generated for specific audio-visual works, 
such as sporting events, will be of finite length. 
Consequently, their corresponding content files will also 
consume a finite amount of storage making it practical to 
perpetually store the entire content file for later viewing. 
However, a traditional television "channel" is composed of 
a never-ending sequence of audio-visual works. Perpetually 
retaining all of the content of a digital channel would 
continuously consume storage at an unacceptably high rate. 
On the other hand, it is desirable to allow users to view 
programs that they may not have been able to view at the 
time the programs were originally broadcast. For example, 
it would be desirable for a viewer to have access to the last 
24 hours of programming that was broadcast over a digital 
channel. According to an embodiment of the invention, 
historical viewing for an infinite feed is provided through the 
use of a continuous finite buffer, where older data "expires" 
and is overwritten with new data. 
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CONTENT EXPIRATION 

In order to have continuous buffer of data, for instance the 
last 24 hours of Lifetime, Television for Women, older 
content needs to be deleted along with the corresponding 
tags. Various approaches may be used to implement such a 5 
continuous buffer. 

With respect to the content data, the simplest approach to 
implement a continuous buffer is to create a single file large 
enough to hold 24 hours of footage. The file is then treated 
as a circular buffer. Specifically, after the creation of the 
initial 24 hour file, the MDS server 110 would establish the 
beginning of the file as the current "insertion point". The 
MDS server 110 would then store new content data over the 
old data at the insertion point, and move the insertion point J5 
to the end of the new data. When the insertion point hits the 
end of the file, it wraps around again to the beginning of the 
file. 

Unfortunately, this single-file circular buffer approach 
makes it difficult to grow or shrink the time of the file. For 2Q 
example, assume that the insertion point is in the middle of 
the file, and a decision is made to expand the file to cover 48 
hours. Under these circumstances, the MDS server 110 
could not begin to extend the time covered for another 12 
hours, when the insertion point would have reached the end 25 
of the file. Using the single circular buffer approach, it is also 
difficult to detect if a client has paused and had the "horizon" 
move over their position, such that when they resume the 
content they were watching has been overwritten. 

FIG. 4 illustrates an alternative, more flexible approach to 30 
buffering a predetermined amount of an infinite video feed. 
Referring to FIG. 4, the content data is stored in a group of 
smaller files 402-414. Each of the smaller files stores a 
subset of the buffered content data. In the illustrated 
embodiment, each of files 402-412 store two hours worth of 35 
content. File 414 currently stores one hour of content. The 
current insertion point is at the end of file 414. When file 414 
reaches two hours of content, file 414 will be closed and a 
new content file will be created. As content files age, the 
older content files are deleted to free up disk space for new 40 
files. During playback, files are joined together seamlessly 
by the video pump as the content data is sent to the client. 

When the buffering technique illustrated in FIG. 4 is used, 
a lenient expiration policy can be set. Specifically, a policy 
may be established that a file is not deleted until all clients 45 
have finished with the (file and any files that precede the 
file). For example, assume that users are allowed to access 
the last 12 hours of a feed. When file 414 is completed, files 
404-414 will contain the most recent 12 hours, so file 402 
is no longer required. However, a client may currently be 50 
viewing the contents of file 402. Consequently, file 402 is 
not immediately deleted. Rather, new clients are prevented 
from accessing file 402, but the client currently accessing 
file 402 is allowed to finish playing file 402. When the last 
client has finished playing file 402, the file 402 is deleted. 55 

To put a cap on the number of existing files, a time limit 
may be established for clients to finish playing old files. For 
example, when file 414 is completed, not only are new 
clients prevented from accessing file 402, but the clients 
currently accessing file 402 are given two hours to finish 60 
playing file 402. At the end of two hours, file 402 is then 
deleted to free up disk space without regard to whether any 
clients were still reading file 402. 

TAG EXPIRATION 65 

When a content file (e.g. file 402) is deleted, the tags that 
correspond to the deleted content file are considered 
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"expired", and therefore can also be deleted. Ideally, tags are 
stored in a format, such as a database table, that allows easy 
deletion of old tags as well as the addition of new ones. 
Unfortunately, the overhead associated with storing and 
retrieving tags from a database table may be too expensive 
to be practical under live feed conditions. For ease and speed 
of access, therefore, tags are typically stored in a flat file. 

Referring to FIG. 5, it illustrates a flat tag file 500. The flat 
tag file 500 includes a header 502 followed by a set of tags 
504. The header 502 indicates information about the con- 
tents of tag file 500, including the set of content files to 
which the tags within tag file 500 correspond. 

As new tags arrive, the tags are appended to tag file 500. 
Because tag file 500 is associated with a continuous feed, tag 
file 500 will grow indefinitely unless a mechanism is pro- 
vided for deleting expired tags. However, tag file 500 itself 
should remain valid even after the expiration of some tags 
(e.g. tags 510) within the tag file 500, since clients may 
continue to access and use the tags 512 within tag file 500 
that have not yet expired. Therefore, the expiration mecha- 
nism cannot simply delete the expired tags 510 from the tag 
file 500. 

Rather than directly delete the expired tags from within 
tag file 500, a temporary tag file 514 is created by construct- 
ing a new header 506 and appending to the new header 506 
a copy of the unexpired tags 512 from the old tag file 500. 
The new header 506 includes the same information as the 
old header 502, except that data within header 502 indicates 
that tag file 500 includes tags for the deleted content file, 
while data within header 506 does not. 

While new tag file 514 is being created, new tag data is 
appended to both the new tag file 514 and the old tag file 
500. After the new tag file 514 is created, new tag data is 
appended to the new tag file 514 rather than the old tag file 
500. To ensure that the new tag data is appended after tag 
data 512, storage for the to-be-copied tags 512 is preallo- 
cated in the new tag file 514, and the new tags are appended 
after the preallocated storage while the existing tags 512 are 
copied into the preallocated storage. 

When all of the unexpired tags 512 have been copied to 
the new tag file 514, the old tag file 500 is closed and the new 
tag file 514 is renamed over the top of the old tag file 500. 
After the new tag file 514 has been renamed, the tag file 
readers (e.g. stream server 118) that were using the old tag 
file 500 are reset based on the information contained in the 
header of the new tag file 514. According to one embodi- 
ment (the "push model"), messages are sent to the tag file 
readers to expressly inform them that the tag file has been 
modified, and that they should update themselves based on 
header information in the new tag file 514. 

According to an alternative "pull model" embodiment, the 
tag file readers are not expressly informed. Rather, they are 
configured to read and update themselves based on the 
header information of the new tag file if they ever fail in an 
attempt to read a tag. The pull model approach has the 
benefit that it avoids the transmission of messages which, 
under many circumstances, are not necessary. 

When tags associated with a particular content segment 
are deleted, clients may continue to view the content seg- 
ment. However, the clients will not be able to perform 
non-sequential access operations that require the deleted tag 
information, such as fast forward and rewind. 

TIMESTAMPING 

Tag information includes timestamp information for each 
of the frames in the corresponding content data. For the 
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purposes of decoding, the timestamp information typically 
represents time relative to the beginning of a feed (i.e. the 
"presentation time"), and is mapped to the byte oflset in the 
content file of the frame that corresponds to that presentation 
time. However, for continuous feeds, such relative time 5 
values are not meaningful. For example, a user would want 
to request playback beginning at Jan. 21, 1997 16:30:23, 
rather than beginning at 5,345,789.76 seconds from the time 
a station began broadcasting. 

According to one embodiment of the invention, absolute 10 
time values are supported by storing an absolute time value 
that corresponds to the "zero" relative time value. Therefore, 
when a client specifies playback from an absolute time, the 
absolute time value associated with "zero" is subtracted 
from the specified absolute time value to yield a relative time is 
value. The relative time value is then used by stream server 
118 to identify the appropriate lag information, and the tag 
information is used by stream server 118 to cause video 
pump 120 to begin sending content from the appropriate 
location in the content file 134. 20 

Typically, the transport formats of digital video provide a 
fixed number of bits (e.g. 33 bits) to represent timestamps. 
For continuous feed environments, the relative timestamp 
values will inevitably reach numbers that cannot be repre- 
sented by the number of bits available in the transport 25 
format. When this occurs, the timestamp values "wrap" and 
begin again at zero. 

To address the wrapping problem, a higher-precision wrap 
value (e.g. 64 bits) is maintained. WheD performing a seek 
or other non-sequential access, the stream server 118 uses 
the higher-precision timestamp values. When transmitting 
content to a client, the video pump 120 sends the lower- 
precision timestamps. 

The video encoding and delivery techniques described 35 
herein empower users with control of functions that were 
previously in the exclusive domain of program providers. 
For example, program providers currently determine which 
plays of a SuperBowl will be replayed to viewers, the speed 
at which they will be replayed, and how many times they 4Q 
will be replayed. 

However, viewers may have strongly differing opinions as 
to which plays merit multiple viewings. For example, two 
viewers may dispute the accuracy of a particular call. 
However, the program provider may not consider the play 45 
that gave rise to the call to be significant enough to replay 
the play. Using the techniques provided herein, viewers may 
determine for themselves which plays should be immedi- 
ately replayed, at what speed they are replayed, and how 
many times they are replayed. 50 

In the foregoing specification, the invention has been 
described with reference to specific embodiments thereof. It 
will, however, be evident that various modifications and 
changes may be made thereto without departing from the 
broader spirit and scope of the invention. The specification 55 
and drawings are, accordingly, to be regarded in an illus- 
trative rather than a restrictive sense. 

What is claimed is: 

1. A method for storing a continuous feed of video, the 
method comprising the steps of: 60 
receiving a digital data stream produced by encoding said 

continuous feed in a digital video format; 
creating a series of content files by repeatedly performing 

the steps of: 

storing said digital data stream in a current file; and 65 
establishing a new file as said current file when said 
current file satisfies a predetermined condition; 



if said series of content files contains more than a prede- 
termined amount of said continuous feed, then deleting 
an oldest content file in said series of content files. 

2. The method of claim 1 further comprising the steps of: 
determining whether any reader is currently playing infor- 
mation from said oldest content file, and 

delaying the step of deleting if a reader is currently 

playing information from 
said oldest content file. 

3. The method of claim 2 wherein the step of delaying is 
performed by delaying the step of deleting until either: 

a predetermined period of time has elapsed; or 
no readers are currently playing information from said 
oldest content file. 

4. The method of claim 1 wherein, if said series of content 
files contains more than a predetermined amount of said 
continuous feed, then preventing new readers from begin- 
ning to access data from said oldest content file in said series 
of content files. 

5. The method of claim 1 wherein the step of establishing 
a new file as said current file when said current file satisfies 
a predetermined condition includes the step of establishing 
a new file as said current file when said current file contains 
a second predetermined amount of said continuous feed. 

6. The method of claim 1 further comprising the step of 
generating and storing tag information that indicates infor- 
mation about frames contained in said digital data stream. 

7. The method of claim 6 wherein the step of storing tag 
information includes the steps of: 

generating a first header for a first tag file; 

storing tag information sequentially in said first tag file; 

when a set of tags within said first tag file becomes 

invalid, performing the steps of 

generating a second header for a second tag file; 

copying all tags in said first tag file, except said set of 
tags, from said first tag file to said second tag file; 

storing new tag information sequentially within said 
second tag file; and 

deleting said first tag file. 

8. The method of claim 7 wherein the step of deleting said 
first tag file comprises the step of renaming said second tag 
file over said first tag file. 

9. A method for providing non -sequential access to video 
from a continuous feed, the method comprising the steps of: 

receiving a digital data stream produced by encoding said 
continuous feed in a digital video format; 

generating tag information that indicates information 
about frames contained in said digital data stream, said 
tag information including timestamps that indicate tim- 
ing of frames relative to a beginning of said digital data 
stream; 

storing an initial time value that indicates an absolute time 
that corresponds to said beginning of said digital data 
stream; 

receiving a request from a client for playback beginning 

at a specified absolute time; 
subtracting said initial time value from said specified 

absolute time to determine a relative time; 
using said tag information to identify a location in said 

digital data stream that corresponds to said relative 

time; and 

transmitting said digital data stream to said client begin- 
ning at said location in said digital data stream that 
corresponds to said relative time. 
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10. The method of claim 9 wherein: 

said digital data stream includes timestamp values having 
a first precision; and 

said step of generating tag information includes generat- 
ing timestamp values that have a second precision, 5 
wherein said second precision is higher than said first 
precision. 

11. A computer-readable medium having stored thereon 
sequences of instructions for storing a continuous feed of 
video, the sequences of instructions comprising instructions 10 
for performing the steps of: 

receiving a digital data stream produced by encoding said 

continuous feed in a digital video format; 
creating a series of content files by repeatedly performing 15 

the steps of: 

storing said digital data stream in a current file; and 
establishing a new file as said current file when said 
current file satisfies a predetermined condition; 
if said series of content files contains more than a prede- 20 
termined amount of said continuous feed, then deleting 
an oldest content file in said series of content files. 

12. The computer-readable medium of claim 11 further 
comprising sequences of instructions for performing the 
steps of: 25 

determining whether any reader is currently playing infor- 
mation from said oldest content file, and 

delaying the step of deleting if a reader is currently 
playing information from said oldest content file. 

13. The computer-readable medium of claim 12 wherein 30 
the step of delaying is performed by delaying the step of 
deleting until either: 

a predetermined period of time has elapsed; or 
no readers are currently playing information from said 35 
oldest content file. 

14. The computer-readable medium of claim 11 wherein, 
if said series of content files contains more than a predeter- 
mined amount of said continuous feed, then preventing new 
readers from beginning to access data from said oldest 40 
content file in said series of content files. 

15. The computer- read able medium of claim 11 wherein 
the step of establishing a new file as said current file when 
said current file satisfies a predetermined condition includes 
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the step of establishing a new file as said current file when 
said current file contains a second predetermined amount of 
said continuous feed. 

16. The computer-readable medium of claim 11 further 
comprising sequences of instructions for performing the step 
of generating and storing tag information that indicates 
information about frames contained in said digital data 
stream. 

17. The computer-readable medium of claim 16 wherein 
the step of storing tag information includes the steps of: 

generating a first header for a first tag file; 

storing tag information sequentially in said first tag file; 

when a set of tags within said first tag file becomes 

invalid, performing the steps of 

generating a second header for a second tag file; 

copying all tags in said first tag file, except said set of 
tags, from said first tag file to said second tag file; 

storing new tag information sequentially within said 
second tag file; and 

deleting said first tag file. 

18. The computer-readable medium of claim 17 wherein 
the step of deleting said first tag file comprises the step of 
renaming said second tag file over said first tag file. 

19. Asystem for delivering a continuous feed of video, the 
system comprising: 

one or more storage devices; 

a video server that stores said continuous feed of video in 
a series of files on said one or more storage devices; 

a threshold detection mechanism configured to detect 
when said series of files holds more than a predeter- 
mined threshold amount of said continuous feed; and 

an expiration mechanism that deletes an oldest file of said 
series of files in response to said threshold detection 
mechanism detecting that said series of files holds more 
than said predetermined threshold amount of said con- 
tinuous feed, 

20. The system of claim 19 further comprising a detection 
mechanism configured to detect whether any reader is 
currently accessing said oldest file, wherein said expiration 
mechanism is configured to delay deletion of said oldest file 
if any reader is currently accessing said oldest file. 

***** 
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